CS - 621 Theory Gems November 14 , 2012 Lecture 16 Lecturer : Aleksander Madry Scribes : Chidambaram Annamalai

In the last lecture, we introduced the streaming model. The goal of this model is to help us develop algorithms that are useful in the scenarios where we want to process large amounts of data while having only very limited storage (and processing time) at our disposal. The motivating example here is a network router. This device has an extremely small memory compared to the network traffic that flows through it (in particular, it would be hopeless for it to try to story all that traffic in its memory). However, we still would like it to be able to compute some useful statistics of that traffic (e.g., number of distinct IP destinations requested, the most frequently requested IP address). Of course, at first, one might expect this to be an impossible task, but – as we will see in coming lectures – quite often one can obtain pretty satisfactory solutions. Formally, in the streaming model (cf. Figure 2), we view the data as a long vector – a stream – y = (y1, . . . , yn), where the length n of the stream is known in advance (and very large). The elements yi of this stream belong to a fixed universe U of size m and (without loss of generality) we will assume that U := {1, 2, . . . ,m} and that m ≤ n. Now, the way we can access the data is quite restricted, we are only allowed to have one pass over stream. So, a streaming algorithm during this pass has to accumulate in its (limited) memory, enough information to be able to compute the desired statistic of the data represented by the stream.